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ABSTRACT 

On-orbit  servicing  (OOS)  is  growing  in  importance  for  the  sustainment  of  certain  satellite  systems.  Although  it  is 
more  economical  to  replace  satellites  in  many  cases,  OOS  could  be  beneficial  or  even  critical  for  more  expensive 
satellites  such  as  Space-Based  Laser  and  constellations  such  as  the  Global  Positioning  System.  Some  future  OOS 
missions  including  refueling  and  modular  component  replacement  will  be  highly  autonomous,  but  there  will  still  be 
a  need  for  humans  to  supervise  and  to  recover  when  unexpected  situations  arise.  Non-routine  tasks  such  as  damage 
repair  or  optics  cleaning  will  likely  require  a  more  significant  level  of  human  control.  The  human  interfaces  for 
such  activities  can  include  body  tracking  systems;  three-dimensional  audio  and  video;  tactile  feedback  devices;  and 
others.  This  paper  will  provide  some  insights  into  when  and  at  what  level  human  interaction  may  be  needed  for 
OOS  tasks.  Example  missions  will  be  discussed  and  the  argument  will  be  made  that  human  interfaces  are  important 
even  for  primarily  autonomous  missions.  Finally  some  current  research  efforts  within  NASA,  academia  and  the 
military  will  be  discussed  including  research  being  conducted  in  the  Human  Sensory  Feedback  Laboratory  at 
Wright-Patterson  Air  Force  Base. 


1.  INTRODUCTION 

To  date,  the  National  Aeronautics  &  Space  Administration  (NASA)  has  relied  extensively  on  astronauts  to  perform 
maintenance  and  repair  tasks  in  space.  Indeed  humans  provide  flexibility  and  ingenuity  that  is  not  yet  possible  even 
with  the  smartest  robot  and  often  this  adaptability  has  been  critical  for  on-orbit  servicing  (OOS)  tasks.  Sanding 
humans  into  space,  however,  requires  expensive  life  support  systems  and  poses  a  risk  of  injury  or  death.|j|Thiis 
plans  for  future  Air  Force  OOS  missions  do  not  call  for  humans  in  space.  The  Defense  Advanced  Research  Projects 
Agency’s  (DARPA)  Orbital  Express  program,  for  example,  is  striving  to  perform  basic  servicing  tasks 
autonomously. 

Human  interaction,  however,  will  likely  still  be  needed  for  future  OOS  missions  at  least  in  a  supervisory  role.  Since 
lower  cost  satellites  are  usually  not  cost  effective  to  service,  OOS  will  typically  be  performed  on  the  most  expensive 
or  critical  satellites,  thus  the  stakes  will  simply  be  too  high  to  blindly  trust  an  autonomous  system  in  such  a  complex 
activity.  Should  something  go  wrong  during  the  docking  or  servicing,  supervisory  controls  can  possibly  prevent 
serious  damage  to  these  valuable  assets.  Software  errors,  incorrect  sensor  calibration,  solar  weather,  and  space 
debris  are  just  a  few  of  surprises  to  be  contented  with  during  a  mission.  Such  robustness  is  quite  difficult  or  cost 
prohibitive  to  automate. 

Through  research  in  uninhabited  vehicles,  the  military  understands  the  importance  of  being  able  to  control 
autonomous  entities.  It  is  arduous  and  costly  to  develop  software  that  is  flexible  enough  to  deal  with  all  possible 
situations  that  can  arise,  thus  supervisory  controls  are  available  on  most  autonomous  systems.  But  basic  human 
computer  interfaces  have  often  turned  out  to  be  inadequate  in  certain  situations.  Interfaces,  including  displays 
(visual,  auditory  and  tactile)  and  control  mechanisms,  need  to  be  carefully  designed  to  ensure  that  the  human  is  able 
to  effectively  react  within  the  time  constrains. 

Many  questions  need  to  be  answered  in  developing  human  interfaces  for  on-orbit  servicing.  Will  a  monitor, 
keyboard  and  mouse  suffice  to  prevent  a  mishap  should  the  human  need  to  take  over?  If  so,  how  should  the  display 
be  laid  out?  How  should  color  and  graphics  be  used?  Are  more  advanced  control  devices  needed?  Can 
performance  be  enhanced  by  using  computer-generated  images  superimposed  onto  live  images  (augmented  reality 
discussed  later)?  To  answer  these  questions  a  designer  may  want  to  match  the  control  requirements  with  the 
human’s  capabilities  and  limitations,  but  this  is  often  not  done. 

For  tasks  that  require  adaptability  like  damage  repair  it  seems  likely  that  humans  will  have  an  even  more  significant 
role  for  the  foreseeable  future.  Tele-operation  of  robots  will  allow  many  tasks  normally  performed  by  astronauts  to 
be  performed  by  humans  on  Earth.  However  it  is  critical  to  determine  how  much  situational  awareness  and  what 
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type  of  control  mechanisms  are  best  for  the  task.  Devices  typically  used  for  virtual  reality  such  as  head-mounted 
displays  and  body  trackers  have  been  successfully  employed  on  a  NASDA  (the  Japanese  space  agency)  Engineering 
Test  Satellite  (ETS-VII)  OOS  test  mission.  It  is  widely  agreed,  however,  that  more  research  is  needed  before  we  can 
be  confident  in  our  ability  to  perform  OOS  tasks  remotely.  Task  performance  can  be  improved  by  employing 
autonomy  for  certain  aspects  (such  as  grasping  objects  with  acceptable  force)  but  it  is  not  clear  how  these  issues 
should  affect  the  human  interfaces. 


2.  SUPERVISORY  CONTROL 


By  strict  definition,  an  autonomous  system  is  void  of  human  involvement.  However  the  reality  is  that  systems  we 
consider  autonomous  usually  provide  some  human  oversight.  The  control  may  be  limited  to  aborting  a  whole 
mission  or  aborting  singles  steps,  or  the  human  may  be  allowed  to  pick  from  a  menu  of  suggested  actions.  It  may  be 
necessary  with  some  systems  to  allow  the  supervisor  to  take  over  operation  manually  if  the  automation  is  unable  to 
handle  the  remainder  of  the  task.  In  any  case,  some  control  is  usually  deemed  necessary  if  the  task  is  considered  at 


all  complex,  uncertain  and/or  critical.  Table  1  taken  from  Sheridan  1992  show 
should  be  noted  that  all  but  level  10  require  some  degree  of  .human  involvement, 
human  interfaces  must  be  considered  for  nearly  all  systems. 


Table  1:  Levels  of  Automation 


Level  Action  performed  by  the  computer.  The  computer... 

HIGH  10.  Decides  everything  and  acts  without  human  involvement 
9.  Informs  the  human  only  if  it,  the  computer,  decides  to 
8.  Informs  the  human  only  if  asked  to 
7.  Executes  automatically  then  must  inform  the  human 
6.  Allows  the  human  a  restricted  time  to  veto  before  automatic  execution 
5.  Executes  the  suggestion  if  human  approves 
4.  Suggests  one  alternative 
3.  Narrows  selection  down  to  a  few 
2.  Offers  a  complete  set  of  alternatives 

LOW  1.  Offers  no  assistance:  human  makes  all  decisions  and  performs  all  actions 


In  supervising  autonomous  systems,  it  is  not  only  important  to  identify  what  controls  will  be  provided  to  the  human 
but  the  interfaces  need  to  center  around  the  capabilities  of  the  human.  For  example  if  actions  are  rarely  required  of  the 
human,  the  designer  must  develop  a  method  to  alert  the  user  without  undue  anxiety.  Once  they  are  alerted,  the  user 
may  need  to  rapidly  gain  situational  awareness  in  order  to  react  appropriately.  Thus  bright  flashing  lights  and  sirens 
may  not  be  the  most  effective  method  to  alert  a  supervisor  of  a  problem. 

2.1  Experiences  with  other  autonomous  systems 

The  AirForce  is  trying  lo  automate  some  functions  that  have  typically  been  performed  by  humans  -  even  piloting 
aircraft]1  njAs  is  to  be  expected,  theje  have  been  some  hard  lessons  to  learn  in  the  development  of  such  systems.  One  of 
these  lessons  occurred  on  March  29,  1999  when  a  Global  Hawk  Unmanned  Aerial  Vehicle  (UAV, 
Figure  1  crashed  at  the  South  Range  of  China  Lake  Naval  Weapons  Center,  California.  The  mishap  occurred  when 
the  highly  autonomous  UAV  inadvertently  received  a  test  signal  for  flight  termination  from  a  test  range  on  Nellis  Air 
Force  Base,  Nevada,  which  was  outside  the  frequency  coordination  zone  in  which  the  UAV's  mission  was  being 
flown.  This  caused  the  UAV  to  go  into  a  termination  maneuver  involving  a  pre-programmed,  rolling,  and  vertical 
descent  from  an  altitude  of  41,000  feet. 
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Figure  1:  Global  Hawk  Unmanned  Aerial  Vehicle 


One  of  the  conclusions  of  the  accident  investigation  was  that  the 
ground  controller  lacked  the  ability  to  override  the  termination 
signal.  The  controllers  on  duty  during  this  incident  indicated 
that  there  would  have  been  sufficient  time  to  override  , 
termination  signal  if  they  would  have  had  the  ability  to  do  so 


£ 


In  another  incident  on  April  18,  1999,  an  RQ-1  Predator  UAV 
crashed  near  Tuzla  Air  Base,  Bosnia  as  a  result  of  both 
mechanical  problems  and  human  factors.  The  Predator  was 
returning  from  a  reconnaissance  mission  over  Kosovo  in  support 
of  Operation  Allied  Force.  According  to  the  accident 
investigation  board  report  prepared  by  Air  Combat  Command 
officials,  the  Predator  experienced  a  fuel  problem  during  its 
descent  into  Tuzla. 


The  two  Predator  pilots,  who  controlled  the  aircraft  from  a  ground  station,  executed  critical  action  procedures  but  were 
unable  to  land  the  aircraft  safely.  According  to  the  report,  the  pilots  became  too  focused  on  flying  the  aircraft  in  icing 
and  weather  conditions  they  had  rarely  encountered.  In  addition  a  lack  of  communication  between  the  two  pilots 
during  the  flight  emergency  was  cited  as  a  cause  of  the  accident. 

2.2  Relating  to  autonomous  satellite  servicing 

Similar  incidents  can  be  envisioned  in  the  performance  of  an  on-orbit  servicing  mission.  If  the  servicer  received  an 
unexpected  signal  from  another  satellite  or  if  the  servicer  collides  with  space  debris,  the  target  satellite  could  get 
damaged,  ruined  or  put  into  an  undesired  spin.  Even  with  supervisory  control  such  events  can  occur,  but  based  on 
previous  experiences,  such  measures  could  mitigate  the  risk. 


These  risk  reducing  measures  could  be  important  for  missions  like  those  anticipated  for  the  U.S.  Defense  Advanced 
Research  Projects  Agency’s  (DARPA)  Orbital  Express.  The  first  mission  for  this  program  is  slated  to  demonstrate  the 
ability  to  transfer  fuel  between  two  satellites.  In  a  possible  follow-on  mission,  refueling  may  be  carried  out  on  the 
Spacj 
year, 
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ased  Laser  Integrated  Flight  Experiment,  which  is  scheduled  for  launch  in  2012  and  testing  the  following 


3.  FUTURE  OF  TELEOPERATION  FOR  SERVICING 


Although  it  is  considered  highly  desirable  to  automate  most  servicing  tasks,  human  control  is  advantageous  for  certain 
tasks.  Souue  tasks  -  such  as  repairs  lor  workarounds  -  may  be  unique,  unpredictable  or  simply  too  expensive  to 


e0 


automate .[TTeleoperation  may  be  the  Best  option  in  these  cases. 


3.1  Applying  new  interface  technologies 

Performing  on-orbit  servicing  tasks  from  a  ground  station  is  challenging  for  several  reasons.  Obviously  a  person  will 
not  have  the  same  level  of  situational  awareness  or  dexterity  as  they  would  when  performing  ground-based 
maintenance  with  their  bare  hands.  However  stereoscopic  displays  -  visual  and  auditory  -  and  haptic  feedback  - 
touch  and  force  -  can  make  up  for  some  of  the  loss  in  situational  awareness. 


Another  significant  challenge  is  telemetry  delays  of  an  estimated  2  to  5  seconds  usually  make  it  necessary  to  work  in 
discrete  steps.  Predictive  displays  have  been  shown 
Technology,  NASA  Jet  Propulsion  Laboratory  and  others. 


s 


ease  these  delay  effects  by  Massachusetts  Institute  of 


Virtual  and  augmented  reality  technologies  may  revolutionize  the  ability  to  perform  remote  servicing.  Body  tracking 
devices,  head-mounted  displays,  haptic  feedback  devices  and  three-dimensional  audio  are  among  the  technologies  that 
can  be  employed.  The  Air  Force  Research  Laboratory,  The  Ohio  State  University,  Wright  State  University  and  Air 
Force  Institute  of  Technology  plan  (in  2002)  to  explore  the  .utility  of  a  virtual  reality  room  called  the  Cave  Automated 
Virtual  Reality  Environment  (CAVE)  for  servicing  activity. 


Remote  servicing  is  also  well  suited  for  the  application  of  augment  reality  in  which  computer-generated  information  is 
superimposed  on  the  real-world  view.  Since  the  precise  location  of  the  cameras  can  be  tracked  with  respect  to  the 
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satellite  being  serviced,  it  theoretically  possible  to  align  the  superimposed  images  with  the  real-world  views.  Thus  the 
scene  can  be  annotated  with  instructions,  part  labels,  virtual  x-rays  or  other  task  guides.^This  capability  can  also  be 
used  with  predictive  displays  where  annotations  are  superimposed  over  virtual  images  rather  than  real  images. 

3.2  Ranger  program 

The  Space  Systems  Laboratory  at  the  University  of  Maryland  in  cooperation  with  NASA  has  been  designing  Ranger,  a 
new  class  of  highly  capable  space  robot.  Ranger  is  designed  to  have  the  ability  to  perform  many  required  operational 
tasks  including  on-orbit  refueling,  instrumentation  package  replacement,  and  deployment  of  failed  mechanisms  such  as 
antennae  and  solar  arrays.  Ranger  has  been  designed  to  be  a  general-purpose  servicing  vehicle,  capable  of  approaching 
a  wide  variety  of  tasks. 

Ranger  was  designed  to  have  force  and  reach  capabilities  similar  to  those  of  an  astronaut  in  a  space  suit.  Ranger  has 
four  manipulators.  These  are  attached  to  the  manipulator  module,  which  is  a  12-inch  cubical  structure  at  the  front  of 
the  robot.  Ranger  has  two  dexterous  manipulators,  a  video  manipulator  and  a  positioning  leg.  Two  dexterous  arms 
enable  cooperative  manipulation,  or  one  arm  can  be  used  to  help  stiffen  the  dock  to  the  work  site  when  large  forces  are 
required.  A  video  manipulator,  in  conjunction  with  wrist  and  boresight  cameras,  help  provide  the  wide  range  of  views 
necessary  when  considering  a  wide  range  of  tasks.  Finally,  a  positioning  leg  allows  the  vehicle  to  be  repositioned  with 
respect  to  the  work  site,  which  is  important  because  of  the  natural  workspace  limitations  of  the  dexterous 
manipulators. 

The  first  on-orbit  demonstration  of  a  Ranger  vehicle  is  scheduled  for  2003  when  the  Ranger  Telerobotic  Shuttle 
Experiment  (TSX,  Figure  2)  will  fly  on  the  Space  Shuttle.  Ranger  TSX  is  designed  to  blur  the  line  of  distinction 
between  human  and  robotic  OOS  operations  by  demonstrating  the  ability  of  a  space  robot  to  perform  some  tasks 
originally  designed  for  human  EVA  subjects,  along  with  tasks  designed  specifically  for  robots.  One  type  of  task  that 
Ranger  TSX  will  perform  is  Orbital  Replacement  Unit  (ORU)  removal  and  installation.  ORUs,  typically  some  form  of 
replaceable  electronics  module,  have  often  been  used  in  the  past  in  human  extra- vehicular  activity  (EVA)  operations, 
and  are  also  planned  for  use  in_future  robotic  operations.  Robotic  ORUs  have  special  fixtures  and  releases  designed 
for  a  robot  to  grasp  and  actuate. 

A  portion  of  the  Ranger  TSX  mission  will  be  to 
demonstrate  the  removal  and  installation  of  human  EVA 
and  robotic  ORUs.  A  task  board  launched  with  Ranger 
TSX  will  include  a  Hubble  Space  Telescope  (HST) 

Electronics  Control  Unit  (ECU)  ORU,  which  was  designed 
for  human  servicing,  and  a  Remote  Power  Control  Module 
(RPCM),  which  is  an  external  component  of  the 
International  Space  Station,  designed  to  be  serviced  by 
robots.  Ranger  TSX  will  approach  these  tasks  using 
methods  similar  to  those  employed  by  human  EVA 
subjects.  A  major  difference  is  that  instead  of  grasping 
EVA  tools  by  hand,  Ranger  will  use  a  set  of 
interchangeable  robotic  end  effectors  such  as  a  bare  bolt 
driver,  a  right  angle  bolt  driver,  an  end  effector  designed 
for  grasping  a  microconical  robotic  interface,  and  various 
grippers.  The  robot  will  be  teleoperated  by  a  human  pilot 
from  the  aft  flight  deck,  or  from  a  ground  station.  The  end 
effectors  may  be  autonomously  interchanged  at  the 
command  of  the  pilot. 

The  first  Ranger  vehicle  (the  predecessor  of  Ranger  TSX)  is  the  Ranger  Neutral  Buoyancy  Vehicle  (NBV,  Figure  3). 
Ranger  NBV,  designed  for  operation  in  the  weightless  environment  created  underwater  using  neutral  buoyancy 
simulation,  has  a  similar  robotics  package  to  Ranger  TSX,  however  it  also  contains  a  full  propulsion  module.  By 
combining  current  robotic  technology  with  a  free-flying  spacecraft  bus,  Ranger  embodies  a  new  class  of  self-contained 
OOS  vehicles  that  will  help  meet  the  demand  for  future  space  operations. 
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Ranger  NBV,  operational  since  its  rollout  in  late  1994,  has 
successfully  performed  a  significant  number  of  realistic 
tasks,  some  unassisted,  and  some  in  cooperation  with  suited 
EVA  subjects.  An  extensive  range  of  experience  has  been 
gained  in  both  free-flying  and  attached  servicing  operations. 

When  in  free-flight,  it  is  often  difficult  to  determine  the 
location  of  Ranger  with  respect  to  the  surrounding 
environment.  Boresight  cameras  cover  only  a  limited  field 
of  view,  and  offer  no  information  about  objects  to  the  sides 
of  the  vehicle.  This  information  is  critical  when 
maneuvering  the  vehicle  in  tight  spaces  in  the  case  that  a 
well-defined  docking  target  is  not  visible.  Typically,  in  this 
case,  an  additional  view  is  desirable.  This  can  be  provided 
by  the  video  manipulator  or  an  external  source  such  as 
cameras  mounted  on  the  target  vehicle. 

When  working  with  robotic  manipulators,  the  cameras  and 
the  operator  are  often  focused  almost  exclusively  on  the  end 
effector  being  controlled.  In  a  multi-armed  vehicle,  it  is 
important  to  also  be  aware  of  the  overall  configuration  of 
the  manipulators  with  respect  to  the  vehicle  and  each  other. 
This  is  important  for  effectively  utilizing  the  workspace  of 
the  manipulators,  avoiding  singularities,  and  also  for 
avoiding  collisions  between  manipulators,  and  with  the 
vehicle  itself.  This  information  is  provided  to  the  Ranger 
NBV  pilot  by  using  the  joint  angles  that  are  passed  back 
from  the  vehicle  to  create  a  computer  generated 
representation  of  the  vehicle  and  the  manipulators. 

Figure  3:  Ranger  NBV 

Additionally,  when  working  with  manipulators,  a  single 
camera  view  of  the  task  environment  has  proven  to  be  rather 
inadequate  for  many  tasks.  It  is  often  significantly  difficult  with  only  one  camera  to  determine  the  distance  between 
the  task  and  the  end  effector,  or  the  attitude  of  the  end  effector  with  respect  to  the  task.  It  is  often  helpful  to  use  a 
stereo  pair  of  cameras  along  with  some  form  of  stereo  display  to  help  determine  the  distance  between  various  objects 
in  the  task  environment.  The  two  forms  of  stereo  display  used  by  Ranger  NBV  are  head  mounted  'virtual  reality'  style 
helmets,  or  alternately,  Liquid  Crystal  Display  (LCD)  glasses  used  in  conjunction  with  a  stereo  interlaced  video 
monitor.  Even  more  helpful  is  the  inclusion  of  an  orthogonal  view  of  the  end  effector.  Clear  and  controllable  views  of 
the  task  environment  are  also  very  important  in  handling  unexpected  anomalies.  Occasionally,  the  task  hardware  does 
not  behave  as  expected,  and  a  clear  view  of  the  environment  with  the  ability  to  inspect  from  various  angles  has  proven 
to  be  invaluable  for  anomaly  resolution. 

Over  its  years  of  operation,  Ranger  NBV  has  replaced  ORUs,  removed  and  attached  electrical  connectors,  assembled 
structures,  opened  access  doors,  removed  fasteners,  walked  "hand  over  hand"  on  EVA  handrails,  and  interacted  with 
EVA  subjects,  both  from  the  end  of  the  remote  manipulator  system,  and  from  free-flight.  This  experience  has  clearly 
demonstrated  the  capability  of  a  trained  pilot  teleoperating  a  highly  capable  dexterous  robotic  vehicle  to  accomplish  a 
wide  variety  of  tasks  under  a  wide  range  of  conditions. 


3.3  Human  Sensory  Feedback  Laboratory 

At  the  HSF  (Human  Sensory  Feedback)  Laboratory,  Wright-Patterson  Air  Force  Base,  Dayton,  Ohio,  an  ongoing 
research  effort  involves  the  investigation  of  issues  related  to  a  cleaning  task  with  a  space-based  laser.  The  ultimate 
goal  in  this  simulation  is  to  incorporate  as  much  autonomy  as  possible,  but  allow  the  human  to  intervene  if  an 
unplanned  emergency  should  occur.  Some  of  the  first  issues  under  consideration  are  the  control  of  such  a  system  in  the 
presence  of  time  delays,  which  typically  occurs  when  the  task  is  at  a  very  distance  location.  For  the  initial  study, 
humans  control  the  cleaning  or  maintenance  operation  at  the  space  based  laser  from  an  outlying  location  and  are  given 
three  display  conditions  with  three  levels  of  time  delays.  Figure  4  illustrates  the  basics  of  the  concept  of  interest.  The 
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three  display  conditions  presented  to  the  human  operator  are  a  direct  view,  a  dual  camera- vie  wing  scenario,  and  a 
virtual  reality  display  (Figure  5). 


Figure  4:  The  Space-Based  Laser  cleaning  simulation 


a 
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Figure  5:  Experimental  design  conditions  for  initial  study 

The  three  levels  of  time  delay  are  zero  seconds,  a  one-second  delay  and  two  seconds  of  time  delay.  The  primary 
objective  of  this  study  is  to  see  if  an  improved  display  condition  could  enhance  operator  performance  of  the  mirror 
cleaning  task,  even  in  the  presence  of  large  time  delays. 

Certain  important  performance  measures  will  evaluate  the  efficacy  of  the  different  display  conditions.  The 
performance  measures  include:  the  time  to  complete  the  task,  minimizing  the  force  interaction  of  the  robotic  system 
with  the  mirror  during  contact,  and  a  measure  of  the  accuracy  of  the  movement  commands  produced.  As  mentioned 
earlier,  autonomy  is  an  important  consideration  in  the  design  of  this  complex  system.  By  incorporating  more  and 
more  levels  of  autonomy  (intelligence)  at  the  end-effector,  it  is  desired  to  demonstrate  better  performance,  especially 
when  the  time  delays  heighten  and  there  is  increased  uncertainty  in  the  operating  environment.  The  ultimate  goal  will 
be  maximum  autonomy  at  the  end-effector,  but  with  the  requirement  that  a  human  can  enter  the  system  in  the  event 
that  mission-critical  events  occur  that  were  not  planned.  Figure  6  illustrates  the  spaced-based  laser  mirror  being 
contacted  by  a  Puma-260-like  robot  in  the  HSF  Laboratory.  Figure  7  illustrates  an  operator  working  in  the  direct  view 
mode. 


6 

SPIE  -  The  International  Society  for  Optical  Engineering 


Figure  6:  Puma-like  robot  simulating  cleaning  task 


Figure  7 :  Operator  in  direct  view  operating  condition 


Figure  8  portrays  another  operator  with  the  camera  and  virtual  reality  view.  The  operator  manipulates  a  “space  ball” 
controller  (figure  9)  that  provides  6  degrees  of  freedom  (three  translations  and  three  rotations).  The  present  level  of 
autonomy  (or  intelligence)  at  the  end-effector  uses  the  concept  of  “force  accommodation.”  This  autonomy 
methodology  means  that  a  local  loop  is  running  continuously  and  maintains  the  end-effector  perpendicular  to  the 
mirror  at  all  times  (independent  of  any  inputs  from  the  human).  This  type  of  autonomy  at  the  end-effector  also 
maintains  and  regulates  a  constant  force  contact  during  the  cleaning  operation.  Local  intelligence  is  thus  provided  and 
an  efficient  means  of  performing  the  task  is  accomplished.  This  procedure  also  unloads  any  commands  from  the 
human  necessary  to  provide  both  proper  contact  orientation  and  regulation  of  constant  contact  force.  Thus  the  human 
has  his  workload  reduced  and  the  autonomy  introduced  at  the  end-effector  has  been  productive  in  the  mission  of 
cleaning  the  mirror. 


Figure  8:  Operator  with  virtual  reality  and  dual  camera  views 


Figure  9:  The  Space  Ball  Controller  by  Force  Sensor 


The  initial  study  in  the  HSF  Laboratory  is  concerned  with  which  type  of  display  condition  may  be  beneficial  to  the 
operator  who  has  to  function  in  environments  where  large  time  delays  may  occur.  Figure  10  is  a  close  up  view  of  the 
virtual  reality  display.  This  is  a  predictive  display,  in  that  it  provides  to  the  operator  an  extrapolation  of  where  the  end- 
effector  is  headed.  Both  velocity  and  position  predictions  are  added  to  this  visual  rendering  in  an  effort  to  improve  the 
operator’s  ability  to  perform  the  cleaning  task  when  large  delays  may  be  present. 
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Figure  10:  Multiple  view  display 


Figure  1 1 :  Method  of  rendering  predictive  display  conditions 


In  Figure  11,  a  description  of  how  the  prediction  is 
rendered  is  portrayed.  The  velocity  of  the  end- 
effector  is  represented  as  the  pink  line;  the  predicted 
position  of  the  end-effector  is  displayed  as  the  black 
line.  By  dynamically  moving  the  robot  (through  the 
interaction  with  the  space  ball  controller),  the 
operator  can  sense  a  projection  of  his  motion  through 
the  action  it  produces  at  the  end-effector.  The  true 
measure  of  success  is  dependent  on  the  performance 
measures  obtained. 

The  goal  of  this  initial  study  is  to  see  if  a  certain 
display  condition  may  be  beneficial  when  the 
operator  has  to  perform  the  cleaning  task  with  time 
delays.  We  have  compared  preliminary  performance 
data  of  the  three  display  conditions  (direct  view, 
camera  view  and  virtual  reality  view)  for  the  three 
delay  conditions  (0,  1  and  2  seconds  of  time  delay). 
The  first  performance  metric  considered  is  based  on 
the  time  to  complete  the  task.  Early  results  indicate 
that  the  degradation  of  performance  due  to  time 
delays  seems  mitigated  by  the  addition  of  the 
predictive  display  condition. 

5.  CONCLUSIONS 

Whether  00  S  activity  is  performed  with 
teleoperation  as  on  the  Ranger  or  autonomously  as 
planned  for  the  Orbital  Express  program,  the  role  of 
the  human  should  not  be  an  afterthought.  Human 
factors  needs  to  be  a  consideration  throughout  the 
system  design  process  because  these  issues  can 
impact  many  aspects  of  the  system.  Experience 
gained  working  with  unmanned  air  vehicles  has 
shown  us  that  overconfidence  in  the  ability  to 
perform  tasks  autonomously  can  lead  to  disastrous 
consequences.  Columns  of  numbers  may  provide 
useful  information  to  the  engineer  who  designed  the 
system  but  when  an  emergency  situation  arises,  more 
elaborate  interfaces  can  mean  the  difference  between 
success  and  failure. 
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